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ABSTRACT: It is well established that a student's capacity to regulate his or her own learning is a 
key determinant of academic success, suggesting that interventions targeting improvements in 
self-regulation will have a positive impact on academic performance. However, to evaluate the 
success of such interventions, the self-regulatory characteristics of students need to be 
established. This paper examines the self-regulatory characteristics of a cohort of second-year 
allied health students, using the evaluation of responses to "meta-learning" assessment tasks 
supported by access data from the learning management system. Students primarily report using 
learning strategies from the performance and self-reflection phases. Although few reported 
using forethought strategies, access to preparatory course materials suggests that these were 
under-reported. Students who reported reviewing lectures as a learning strategy were more 
likely to access the online lecture recordings; however, higher access was associated with poorer 
academic performance. Cluster analysis of all available data showed high academic performance 
was positively associated with early submission of intra-semester assessment tasks but 
negatively associated with both use of, and reported of use of lecture recordings by students. 

These findings suggest that early submission of intra-semester assessment may be useful as a 
predictor of academic achievement. 

KEYWORDS: meta-learning, self-regulation, allied health 

1 INTRODUCTION 

As knowledge increases with often overwhelming complexity, the development of lifelong learning skills 
is an imperative for graduates to excel in a global society. These skills are particularly important in 
professions such as science and the allied health sector, where the pace of new knowledge generation is 
rapidly accelerating. While it is well established that a student's capacity to self-regulate learning is a key 
determinant of academic success and the ability to overcome academic adversity (Turner & Husman, 
2008; Zimmerman, 2002), the development of self-regulatory skills is also considered critical to the 
development of lifelong learning (Schunk, 2005). 

Self-regulated behaviour can be defined as the thoughts, feelings, and actions planned and adapted by 
an individual in order to attain a self-selected goal (Zimmerman, 2000). Self-regulated learning involves 
more than just detailed knowledge; it encompasses the self-awareness, motivation, and behavioural 
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adjustments made in order to implement knowledge. There are various theories of self-regulated 
learning with differing levels of complexity (Winne, 1996; Boekaerts, 1997; Zimmerman, 2000; Bannert, 
Reimann, & Sonnenberg, 2013). Most theories agree that self-regulated learning is adaptive and is 
therefore cyclical in nature. Learners modify their learning strategies to suit the task at hand and 
through critical appraisal of past learning events. The current study utilizes Zimmerman's three-phase 
cyclical model of self-regulation (Zimmerman, 2000; Nota, Soresi, & Zimmerman, 2004). This well- 
established model proposes that self-regulation occurs through three strategic phases: forethought, 
performance, and self-reflection (Zimmerman, 2000). 

The forethought phase involves processes, beliefs, and thoughts that occur prior to learning. Processes 
in this phase include task analysis, involving goal setting and strategic planning, and self-motivation 
(Zimmerman, 2002; Cleary & Zimmerman, 2004; Schunk, 2005). The performance phase involves 
behaviours implemented in response to the learning process (Postholm, 2011), with the main processes 
being self-control and self-observation (Zimmerman, 2000). Finally, the self-reflection phase occurs after 
learning has taken place. It involves self-reaction and self-judgement processes, with self-evaluation and 
causal attributions being part of the latter process (Dunlosky & Rawson, 2012; Zimmerman, 2000). 
Together, these phases represent the various processes undertaken within self-regulation, and are 
cyclical, as feedback from prior efforts informs adjustments to both current and future learning 
attempts (Zimmerman, 2000). 

In order for students both to recognize and to modify their self-regulatory behaviour, they must develop 
an awareness of their own learning and use this knowledge to control it (Winters, 2011). When students 
learn to determine what causes the performance of a task to be either successful or unsuccessful, they 
are able to go beyond the goal of just being accurate learners as they gain an understanding of the 
conditions under which certain learning strategies are most effective (Vilalta, Giraud-Carrier, & Brazdil, 
2010). The awareness of one's own learning is referred to as meta-learning (Winters, 2011). Essentially, 
meta-learning involves various metacognitive aspects of learning, whereby students are aware of their 
motives, abilities, and the demands of a learning task, and are able to control their behaviour to achieve 
desired outcomes (for review see Jackson, 2004). In an extensive synthesis of over 800 meta-analyses, 
Hattie (2009) found that meta-cognitive strategies have some of the most powerful effects on improving 
student learning. For these reasons meta-learning is often associated with the theory of self-regulation 
(Winne, 2001; Zimmerman, 2006). 

Metacognitive processes are inherently personal and therefore difficult to observe in students. In this 
study of allied health university students, self-reports through "meta-learning" assessment tasks were 
utilized to collect information about self-regulatory behaviour and to evaluate student use of different 
categories of self-regulatory strategies from each phase of the self-regulation cycle. However, as self- 
reported data has potential limitations in accuracy (Hadwin, Nesbit, Jamieson-Noel, Code, & Winne, 
2007; Schmitz & Wiese, 2006), it has been combined with learning analytics of data available from the 
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course learning management system to enhance this understanding of student self-regulatory behaviour 
and processes in a non-research setting. 

The aims of the study were to 1) identify the self-regulatory learning strategies employed by second- 
year allied health students; 2) characterize student behaviour in interacting with course materials, in 
view of their self-reported planning and use of strategies; and 3) compare and contrast the use of these 
different sources of information for evaluating the self-regulatory behaviour of students in an authentic 
setting. 

2 METHODS 

2.1 Institutional and Course Context 

The University of Queensland is a large, research-intensive Australian university, with over 40,000 
undergraduate and 8,000 post-graduate students. The participants for this study were second-year 
undergraduate students in the Bachelor of Physiotherapy (n=121) or Bachelor of Speech Pathology 
(n=95) programs or post-graduate students in the Master of Speech Pathology (n=ll) program. The 
entrance requirements for these programs are very high, although the academic backgrounds of 
students tends to differ between programs (Ernst & Colthorpe, 2007). All the students took a Human 
Physiology course, which covered cell, nerve, and muscle physiology, and the physiology of the 
cardiovascular, respiratory, and renal systems. Students enrolling in the course had an average age of 
20.2 years, 75% were female, and 12% were international students. All enrolled students had access to a 
course Blackboard™ 1 site through which they were invited to participate in the study. No incentive for 
participation was offered. Of the cohort, 99 students (44%) provided informed consent for their 
inclusion in this study, which was approved by the University of Queensland Human Experimentation 
Ethical Review Committee. 

The course consisted of three hours of lectures every week together with three-hour practical classes in 
eight of the thirteen weeks of semester. Lectures were recorded through Echo360™ 2 and were made 
available to students through a folder on the course Blackboard site 1-5 days after the lecture took 
place. Lectures were scheduled twice a week with a 1-hour lecture on Tuesdays and a 2-hour lecture on 
Wednesdays, giving a total of 26 lecture recordings. Seven practical classes were laboratory based and 
one practical class was an online scenario-based learning task completed during scheduled class time. In 
the laboratory-based classes students undertook short experiments, demonstrating physiological 
concepts such as osmosis or excitation-contraction coupling of skeletal muscle. Each laboratory-based 
practical class had an introductory video to aid students' preparation for the class, with one video 
covering a combination of topics from two classes, so that there were a total of six videos. These videos 


1 Blackboard Inc., Washington, DC, USA. 

2 Echo360 Inc., Dulles, VA, USA. 
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were also contained in a folder on the course Blackboard site and were available from the 
commencement of the semester; student access of that folder was tracked. 

Assessment for the course consisted of a mid-semester exam worth 20%, a written assignment worth 
13%, and an end-of-semester exam worth 55%. Further four equally weighted meta-learning tasks, 
collectively worth 12%, were used to assess student management of learning and promote self- 
regulatory behaviour. 

2.2 Meta-Learning Assessment Tasks 

Students completed four meta-learning assessment tasks during the semester at approximately three- 
week intervals. Each task comprised six questions designed to help students articulate their own 
learning and engage in learning strategies from all the self-regulatory phases (Zimmerman, 2000). The 
first meta-learning task was developed with the purpose of determining the goals and motivations of 
students; it asked students to articulate the study strategies they had used in the past and identify 
hindrances to their learning. The second task was completed two weeks prior to the mid-semester exam 
and was developed with the aims of increasing student awareness of their understanding of course 
content, articulating strategies they may use to improve their learning, and promoting effective study 
for the mid-semester exam. Students completed the third meta-learning task after the mid-semester 
exam. It aimed to encourage students to reflect on the strategies they had used for mid-semester exam 
study, to determine the effectiveness of these strategies, and to consider how they could improve their 
study for future exams. The final meta-learning task completed the self-regulatory cycle by allowing 
students to reflect once again on their learning and propose strategies they could use to assist their 
study for the upcoming end-of-semester exam (Zimmerman, 2000). Students were awarded a small 
number of marks (0.5% per answer) if the answer was appropriate and relevant, with the vast majority 
of students who completed the task receiving full marks. Each meta-learning task was available for one 
week on the course learning management system, and students could access and submit their 
completed task at any time during that week. The date and time of each of their submissions was 
recorded. 

2.3 Qualitative and Quantitative Analysis 

To identify the self-regulatory strategies students reported using, the first meta-learning task included a 
question asking students to list all the learning strategies they employed. The responses of consenting 
students were then categorized, based on the classification system of Nota et al. (2004) adapted to suit 
the university setting (Table 1). These adaptations included the following: 1) splitting the "goal setting 
and planning" category to identify student goals and the strategies they planned to use to achieve those 
goals; 2) expanding the "seeking social assistance" category to include collaborative learning; and 3) 
splitting the "organization and transforming" category into two. "Organization" was defined as time and 
resource management. "Transforming" was expanded to include actively reappraising records and 
summarizing them into various forms of information, such as concept maps, lists of key points, and 
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diagrammatic representations. Where a student response encompassed more than one category, that 
response was coded to all relevant categories. Blind coding was initially performed by one researcher; a 
subset (33%) of responses was then blindly coded by a second researcher, demonstrating an inter-rater 
coding reliability of 82%. Once coding was complete, the number and type of strategies reported by 
each student were used for comparison to academic performance and learning analytics data. 

Across the four meta-learning tasks, the students were asked a series of questions probing their self¬ 
regulation strategies. Students reported on 1) strategies they had used in the past; 2) strategies they 
intended to use to study for the mid-semester exam; 3) strategies they had used to study for that exam; 
and finally 4) strategies they intended to use to study for the end-of-semester exam. Students were 
categorized on whether they did or did not mention using lecture recordings in their study in at least 
one meta-learning task. 

Overall, students had five intra-semester assessment tasks submitted through the learning management 
system. Four of the meta-learning tasks were open for submission for one week. The fifth item was an 
assignment, with topics provided in week 3 of semester, and submissions due in week 11. The 
submission date and time for each student was collected from the learning management system. 

Throughout this study, quantitative analyses were performed using the GraphPad Prism 6™ 3 or R 3.1.1 4 
programs. The results were expressed as mean and standard error of the mean (SEM), and were 
considered significant if p<0.05. Where possible, anonymous aggregate data from the whole cohort was 
used for analyses, but where comparisons of qualitative data to academic performance or access to 
resources were performed, only de-identified data from consenting students were used. The end-of- 
semester exam results of consenting (n=99, 44% of the cohort) and non-consenting students (n=127) 
were subjected to an unpaired t-test. The mean end-of-semester exam performance for the consenting 
students (73.4 + 1.2) was not significantly different from that of the whole cohort (71.7 + 0.8), 
suggesting that the consenting students are representative of the range of academic performance levels 
present in the cohort. 

Exploratory cluster analysis was used to determine if there were differences in academic performance 
(course grade) between groups of consenting students who shared approaches to the regulation of their 
learning, extent and timing of lecture recording use, and timing of assessment submission. In stage 1, 
Ward's (1963) hierarchical method was used to produce a cluster dendrogram (Figure 1). The clearest 
demarcation partitioned students into two clusters. In stage 2, the k-means method with Euclidean 
distances was used to explore the characteristics of partitioning students into 5, 3, and 2 cluster 
groupings. Reducing k-means from 5, to 3, to 2 dramatically improved the clarity of the relationship 
between cluster allocation and academic performance. In the final k=2 clustering, the cluster of high 
performers scored 80.2+/-0.9% (n=81 students) while the cluster of low performers scored 68.3+/-1.8% 
(n=13 students). 


3 San Diego, CA, USA. 

4 R Development Core Team, Auckland, NZ. 
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Table 1. Strategy classifications, definitions (adapted from Nota et al., 2004), phases (Zimmerman, 
2000), and example student responses from meta-learning tasks. 


Self-Regulation Phase 

Strategy 

Definition 

Examples from Student Responses 

Forethought 

Goal Setting 

Setting goals or sub¬ 
goals (student 
initiated). 

"Set smaller and more manageable 
goals over a long period of time (go 
through a few concepts a day, not the 
whole 3 or 4 weeks of lectures in a 
few hours)." 

Strategic Planning 

Developing plans to 
use specific learning 
strategies or 
behaviours. 

"In the past, 1 would review lecture 
notes and take notes in lectures. 1 
plan on doing the same this semester 
with an even greater emphasis on 
taking notes." 


Environmental 

Structuring 

Adapting physical or 
virtual surroundings 
to be conducive to 
learning. 

"1 definitely use the private study 
section of the library to remove 
distractions." 


Keeping Records 

Recording events or 
results (student 
initiated). 

"Listening attentively in lectures and 
taking down relevant points which 
would aid my revision." 


Organisation 

Managing time and 

resources 

appropriate to the 
task. 

"Organise set times to complete 
everything by a set time or date (like 
study for a set amount of time, and 
then if done 1 allow myself to go out 
with friends the next day etc.)" 

Performance 

Reviewing 

Records 

Re-reading notes or 
supplied resources, 
or accessing lecture 
recordings. 

"Regularly reviewing my lecture notes 
so that I'm not learning it all for the 
first time just before exams." 


Seeking 

Information 

Securing further 
relevant 

information from 

non-social sources. 

"Watching YouTube videos or 
researching concepts to get concepts 
explained in different ways." 


Seeking Social 
Assistance 

Learning 

collaboratively with 
others or seeking 
help from peers, 
lecturers, and 
others 

"Discussing with friends and asking 
them questions, explaining to each 
other all the different concepts and 
principles." 

Self-Reflection 

Active 

Reappraisal or 
Transformation 

of Records 

Appraising and 
rearranging records 
or resources to 

improve learning. 

"1 write notes and summaries for each 
lecture, lastly 1 draw concept maps, 
diagrams, or flow charts to better 
visualise the information learnt during 
the lectures." 


Self-Evaluation 

Evaluating quality or 
progress of learning 
or effectiveness of 
strategies used. 

"Doing past year exam questions 
which reinforces my knowledge and 
alerts me to what 1 do not know." 
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Figure 1. Cluster dendrogram of consenting students (n=99) based on access of lecture recordings, 
meta-learning and assignment submission dates, and assignment and course grades. Height 
represents the degree to which clusters of students can be differentiated based on the variables 

used for analysis. 


3 RESULTS 

Students (n=97) reported that they previously used an average of 5.3 + 0.2 learning strategies, with 
students reporting between 1 and 15 strategies each. A total of 511 responses were coded, with reports 
of strategies from across all 10 categories. The most commonly reported category was "reviewing 
records" (71% of students), which included reviewing lecture recordings. The least reported category 
was "goal setting," reported by only 1% of students. All students reported using strategies from the 
performance phase of the self-regulatory cycle, 68% also used strategies from the self-reflection phase, 
but only 8% reported strategies from the forethought phase (Figure 2). Analysis by unpaired t-test with 
Welch correction showed that students who reported the use of strategies from more than one phase of 
the self-regulatory cycle (n=73) performed significantly better in the course overall (79.1 + 1.08%) than 
students who reported using only strategies from the performance phase (74.3 + 1.68%). Further 
analyses using Pearson's correlation were performed to determine the relationship between individual 
self-regulatory strategies and overall achievement in the course. There was a small, but significant, 
negative correlation between environmental structuring and academic performance (r = -0.2019, 
p<0.05) indicating that students who reported undertaking environmental structuring more frequently 
were likely to achieve lower academically than those who did not. Surprisingly, there were no other 
significant correlations between the use of individual strategies and overall performance. 

The most frequently reported strategies of self-regulation (90% of students) were reviewing records and 
active reappraisal of records, which included references to using lecture recordings. Across all four 
meta-learning tasks, 48% of students specifically reported viewing lecture recordings as a learning 
strategy, a further 18% said they reviewed lectures but did not specify whether these were lecture notes 
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or recordings, whilst 34% of students did not specifically mention the lecture recordings but did mention 
reviewing lecture notes. Students who mentioned using lecture recordings were twice as likely to report 
using lecture recordings rather than planning to use them. This "reactive" pattern is consistent with the 
low level of planning strategies that students reported (Figure 2). 


120 



Figure 2. Frequency distribution of the categories of self-regulatory strategies (Nota et al., 2004) 
consenting students (n=97) reported they had previously used. Students used an average of 5.3 ± 0.2 
strategies each, ranging between 1 and 15 strategies from the forethought (black bars), performance 
(light grey bars) and self-reflection (dark grey bars) phases of the self-regulatory cycle (Zimmerman, 

2000 ). 

Although only half of the students explicitly stated in meta-learning assessment tasks that they used 
lecture recordings to support their learning, every student in this study accessed the folder containing 
the lecture recordings at some point during the semester. There was an extremely large range in the 
extent of lecture recording use, from a student who accessed lecture recordings on just 3 days of the 
144-day semester, to a student who accessed lecture recordings on 41 days over the semester. Lectures 
for this course were held every Tuesday and Wednesday, and there was a general pattern of increased 
access to lecture recordings between Tuesday and Thursday for every week of semester (Figure 3), 
indicating a consistent use of lecture recordings around the time of each recording. Flowever dramatic 
peaks in access occurred in the days prior to the mid-semester and end-of-semester exams (Figure 3). 

On average, individual students accessed the lecture recordings folder 48.1 + 2.4 times, but the variation 
between students was very large, with individuals accessing the folder anywhere between 4 and 194 
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times. Students who reported reviewing lectures as a learning strategy in meta-learning were 
significantly (p<0.0001) more likely to access the folder more frequently, and on more days, than 
students who did not report using lecture recordings (Table 2). 
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Figure 3. Student (n=227) access of the folder containing lecture recordings. Lectures commenced on 
Tuesday of week one of the semester, and were added sequentially as the semester progressed. The 
folder ultimately contained 26 recordings; two for each week of the semester. The mid-semester 
exam was conducted at the end of week 8 (see arrow). 


Table 2. Comparison of consenting student (n=99) actual access of the lecture recordings and their 
reported use or intention to use lecture recordings as a learning strategy in meta-learning analysis. 
Data were analyzed using a non-parametric Mann Whitney U test. 


Meta-Learning Analysis 




(did and/or intended to use lecture 

n 

Frequency 

Days 

recordings) 




Yes reported 

47 

66.1 + 5.6 

21.3 + 1.3 

Never reported 

52 

28.2 + 2.5*** 

13.4 + 0.9*** 


*** Significant p<0.0001 difference between groups 


The majority of students (71%) reported using learning strategies that involved organizing their learning. 
Some examples of such organizational strategies included modifying their environment, either physically 
("/ definitely use the private study section of the library to remove distractions") or virtually 
("disconnecting the Internet so I don't suddenly find myself online shopping or on Facebook etc.")-, 
managing their time ("Studying for shorter 1-hour blocks, rather than 3-hour blocks. This is to aid my 
focus and attention "); or their behaviour ("making lists of things I need to do”). These represent 
strategies of self-control and self-observation (Zimmerman, 2000) carried out as part of the 
performance phase of the self-regulatory cycle (Figure 2). 
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Of the whole cohort, 65% of students accessed the online folder containing the practical introduction 
videos, with the majority of them accessing it one to three times. The access occurrence was highest just 
prior to the first practical class, in week 3 or 4 of semester, with 56% of students having accessed the 
folder by the time their first class was scheduled (Figure 4), and access declining thereafter. Higher 
frequencies of access tended to occur on Thursdays and Fridays, which corresponds to the scheduled 
time of the classes. While only 7.4% of students accessed the folder six or more times, it was possible to 
watch or download more than one video at any given access, so use is probably greater than this 
percentage indicates. 



,1 


I ■ .1 I .1.11. . jl ....ll 


.11 


Week of Semester 


,x> N <f e 

«y 

& <& 


Figure 4. Student (n=227) access of the online folder containing introductory videos for the laboratory- 
based practical classes. Students commenced practical classes on Thursday afternoon or Friday 
morning of either week 3 or week 4 (at red arrows). The folder contained six videos; five specifically 
for a single class, with a further video containing a combined introduction for two of the classes. 


For each meta-learning assessment task, the date and time of submission was recorded, with the 
average time of submission prior to the due date varying for each task (Figure 5). Students submitted 
the fourth meta-learning task significantly (p<0.05) earlier than the second meta-learning task (Figure 5). 
There were significant (p<0.01), modest, positive correlations between the times students submitted all 
four meta-learning tasks, indicating that the time at which students submit one meta-learning task 
explains 37-53% of the variation in the timing of their submission of their other meta-leaning tasks 
relative to the release data and the due date (Figure 5). In addition, the earliness of a student's meta¬ 
learning submission was significantly positively correlated with his or her course grade (r = 0.25, p<0.05). 
There was also considerable variation in the time that students submitted the assignment: 44% of 
students submitted within the 24 hours prior to the deadline, with a further 30% submitting 24-48 
hours prior. Only a small proportion of students (1.3%) submitted late, but these were within 24 hours 
of the submission deadline (Figure 6). 
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Figure 5. Submission times for meta-learning tasks 1-4 (A, ML1; B, ML2; C, ML3; D, ML4) prior to their respective deadline. Tasks were due at 
approximately three-week intervals, with tasks due on Wednesday of weeks 4, 7,10, and 12 respectively. Submission time for individual students 

showed significant (p<0.01), modest, positive correlations. 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


144 















































JOURNAL OF LEARNING ANALYTICS 


S-LAR 


(2015). Know Thy Student! Combining Learning Analytics and Critical Reflections to Increase Understanding of Students' Self-Regulated 
Learning in an Authentic Setting. Journal of Learning Analytics, 2(1), 134-155. 



Days prior to due date 

Figure 6. Time of submission of assignment prior to the due date by students (n=216). Three 
students submitted after the deadline (two within five minutes of it, and the other 15 hours late). 
The remaining seven students had extensions granted. The maximum time prior to the due date 

that a student submitted was 9.5 days. 

Analysis of student access of the lecture recording data revealed two patterns of access behaviour, 
high access and low access. The access patterns are represented as a heat map on the calendar of 
semester (Figure 7A-B). 


Table 3. Characteristics of high and low performing clusters of students (n=94). 


Cluster Analysis 

High performers Low performers 

(n=81) (n=13) 

Course grade (%) 

80.2 + 0.9 68.3 + 1.8*** 

Lecture recording use 
(number of accesses across 
semester) 

45 +4 62 + 7* 

Self-reported lecture recording use 
(number of meta-learning tasks) 

0.77 + 0.12 1.70 + 0.35** 

Meta-Learning task submission 
(days and hours before deadline 
across four tasks) 

„ . „ _ . . . 21 hours + 3 

4 days 19 hours + 4 hours . 

” hours*** 

Assignment submission (hours 
prior to deadline) 

32.2+ 4.7 hours 23.8 +17.7 hours* 


*p<0.05, **p<0.01, *** p<0.0001 significantly different between clusters 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


145 






















JOURNAL OF LEARNING ANALYTICS 


S8LAR 


(2015). Know Thy Student! Combining Learning Analytics and Critical Reflections to Increase Understanding of Students' Self-Regulated Learning in an Authentic Setting. Journal of Learning Analytics, 
2(1), 134-155. 


2014 



Wednesday 

Ttxrjday 

Friday 

Saturday 


B 


Apr ^ May ^ Jun ^ 

Mid-semester Study Final 

exam break exam 


Wednesday 

Ttxnoay 



^ May ^ Jun ^ 


Mid-semester 

exam 


Study 

break 


Final 

exam 


Figure 7. Student access of lecture recordings for Cluster 1 (A: High Access) and Cluster 2 (B: Low Access) as heat maps on a calendar. The 
darker the blue, the higher the frequency of access. Note that the scale varies from 0-35 on A and 0-12 on B. The semester commenced on 
March 3, with the first lecture on March 4, and the mid-semester exam on the Saturday of week 8 (May 3). The teaching period finished on 
the Friday of week 13 (May 30), and was followed by a one-week study break and the end-of-semester exam period, with the final exam on 

June 25. 
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Closer examination of the two student clusters identified from the overall analysis (Figure 1) revealed 
that the extent to which the two groups accessed lecture recordings differed significantly, with the high 
performers accessing the lecture recordings less frequently than the low performers (Table 3). However, 
there was no appreciable difference in the timing of lecture recording use across the semester for the 
high performing cluster (Figure 7A) or the low performing cluster (Figure 7B). The difference in the 
extent of lecture recording use was consistent with student self-reports (Table 3), with the high 
performing cluster indicating that they used lecture recordings in fewer meta-learning tasks (typically 0- 
1 task) than the low performing students (typically 1-2 tasks). Interestingly, across the four meta¬ 
learning tasks, the low performers were more likely to report that they used the lecture recordings, than 
report that they planned to use the lecture recordings to study for exams. This pattern was not apparent 
for the high performers, who neither reported using, nor planning to use, lecture recordings. Finally, the 
most dramatic differences between the high and low performing clusters related to assessment 
submission times. For all four meta-learning tasks and the assignment, the high performers submitted 
their work significantly earlier than the low performers (Table 3). 

4 DISCUSSION 

The combination of student responses in meta-learning assessment tasks together with student access 
to online data was used in this study to characterize allied health science students in terms of the self- 
regulatory strategies they employ, their organization, and their preparedness for study. However, each 
of these forms of information is limited in providing accurate information on students' self-regulatory 
processes. Some learning processes are particularly difficult to track in a traditionally run, on-campus 
setting; for example, information seeking, seeking social assistance, and environment structuring. Self- 
reporting is particularly useful for providing insight into such processes, which are not always easily 
tracked online in adult learners with clear preferences for certain self-regulatory strategies (for example, 
"Watching YouTube videos for concepts that I don't understand and need to clarify" and "Studying with 
friends in group discussions where we talk about concepts, and take turns explaining things to each 
other, in order to come to a group consensus about more difficult topics"). However, self-reported data 
has well-documented flaws, including variations in the interpretation of the learning context in question, 
and inaccuracies in self-reporting the frequency of strategies (Perry & Winne, 2006). The combined 
analysis of students' online access of data together with self-reporting within meta-learning assessment 
tasks allows a more comprehensive view of student self-regulatory behaviour. Furthermore, using this 
combination of sources of information should allow both a verification of the self-reported student data 
and a greater understanding of the learning behaviours that underlie patterns of student access to 
resources on the learning management system site. For academics in an on-campus, non-research 
environment, the combination of these tools may assist them to increase their understanding of their 
students' self-regulatory processes. 

Analysis of the student responses to the meta-learning tasks showed that students in this study used a 
broad range of self-regulatory strategies, particularly those from the performance and self-reflection 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


147 


JOURNAL OF LEARNING ANALYTICS 


S8LAR 

SOCIETY 1 LEARNING 
ANALYTICS RESEARCH 

(2015). Know Thy Student! Combining Learning Analytics and Critical Reflections to Increase Understanding of Students' Self-Regulated 
Learning in an Authentic Setting. Journal of Learning Analytics, 2(1), 134-155. 

phases of the self-regulatory cycle (Figure 2). The breadth of learning strategies used is likely a reflection 
of the educational experiences of the students. Our study cohort were second-year undergraduates and 
post-graduates in intellectually demanding programs at an advanced stage of their education, so 
consequently they appear to have a broad repertoire of learning strategies. In addition, those who 
utilized strategies from multiple phases of the self-regulatory cycle performed better, suggesting that a 
diversity of strategies is beneficial. The most common strategies reported were the review and 
reappraisal of records, including using both lecture recordings and notes. Many students also reported 
that they structured their environment to be conducive to learning, although their performance was 
likely to be poorer the more frequently they did so. This may reflect that students with very specific 
environmental needs may not be adaptable to the varying learning contexts at university or are easily 
distracted within them. Interestingly, based on the initial questioning of students undertaken here, 
fewer than 10% of students reported using the forethought strategies of planning or setting goals for 
their learning. For example, only a single student reported setting goals as a learning strategy. Flowever, 
this finding needs to be viewed with caution as the responses to the meta-learning questions, while rich, 
are dependent on context. 

Goal setting has previously been identified as an important learning strategy, being closely linked to 
academic achievement, with students who set a combination of mastery and performance goals having 
better academic achievement than those who do not (Sandars & Cleary, 2011; Luo, Paris, Flogan, & Luo, 
2011). Strategic planning is also considered an important self-regulatory process, but is cognitively 
demanding, as it requires learners to understand the area of expertise they wish to acquire, have insight 
into their existing knowledge, and possess pedagogical knowledge to make informed decisions 
(Bonestroo & de Jong, 2012). Given these considerations, strategies from the forethought phase are 
considered both more advanced than those from the performance phase and essential for effective self¬ 
regulation (Postholm, 2011; Schunk, 2008; Turner & Flusman, 2008; Zimmerman, 2008). It is therefore a 
concern that these students did not appear to be engaging in forethought processes. Flowever, in the 
first meta-learning task, students were not specifically asked to identify their goals or describe their 
plans for learning; rather they were asked to describe the learning strategies they had previously used. 
When prompted to undertake forethought processes in the second meta-learning task, by being asked 
their intentions in the period leading up to the mid-semester exam, students clearly demonstrated that 
they could strategically plan their learning, reporting multiple strategies when prompted, but again very 
few mentioned setting goals for their learning. Together, these findings suggest that either generally 
students don't engage in forethought processes unless prompted, or that their forethought processes 
are primarily limited to planning. Alternatively, students may set goals, but not consider goal setting to 
be a "strategy" for learning. 

Students who reported using or intending to use lecture recordings in the meta-learning tasks accessed 
the recordings significantly more frequently and on more days (Table 2). In addition, there were clearly 
defined clusters of student learning behaviour in using lecture recordings (Figure 7), identified both 
within online access and meta-learning data with high prediction accuracy, with students who exhibit 
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extensive use of lecturing recordings having poorer academic performance than those who made less 
use of the recordings (Table 3). This is not surprising if the meta-learning responses of the students are 
considered in more detail. Where students reported simply that they reviewed lectures, regardless of 
the form, these were classified as "reviewing records," which is part of the performance phase of the 
self-regulatory cycle (Zimmerman, 2000; Nota et al., 2004). "Reviewing records" was the most 
commonly reported category, and most reports of lecture recording fell into this category. Whereas, if 
students reported that they engaged with the lecture material more actively — for example, if they used 
the material to create summaries, concept maps, and diagrams of the material, or developed lists or 
questions about key concepts — then these were classified as "active reappraisal of records" (Nota et 
al., 2004). Few students mentioned using the lecture recordings for this purpose. If students actively 
engage with learning materials, when they go beyond simple review of their notes and/or the supplied 
resources, then they are engaging with the material in a more reflective way. In addition, students may 
use the material they develop to test themselves or may use the supplied practice exams. By evaluating 
their knowledge and understanding in these types of strategies, students engage in key processes within 
the self-reflection phase of the self-regulation cycle (Zimmerman, 2000). These processes are more likely 
to have a positive impact on student learning than the simple review of records. 


More than 70% of students reported using learning strategies to organize their learning, particularly in 
terms of creating optimal opportunities for study, by managing their time, their behaviour, or by 
modifying their environment (Figure 2). The use of these strategies reflects that students are monitoring 
and moderating their learning, as part of the performance phase of the self-regulatory cycle. The main 
meta-cognitive processes within this phase are self-control and self-observation, with self-control 
requiring attention to and awareness of one's actions and how they affect outcomes, and self¬ 
observation occurring when students systematically monitor their performance (Zimmerman, 2000; 
Schunk, 2005). Furthermore, engagement in these processes has been previously shown as a significant 
predictor of academic achievement (Kitsantas, Winsler, & Huie, 2008). While these types of 
organizational learning strategies may be difficult to identify solely via tracking student access to 
resources on course learning management system sites, the extent of a students' organization may be 
reflected in their approach to intra-semester assessment tasks. Therefore the punctuality or "earliness" 
of the assessment submission was analyzed (Figures 5 and 6) and compared to academic achievement 
and reporting of organizational strategies (Table 3). There was a significant relationship between the 
times students submitted the meta-learning assessment tasks and assignment with the overall course 
grade (Table 3). Flowever, when comparing the self-reporting of organizational strategies by students, 
there was no relationship with course grade, except for environmental structuring, which had a small 
negative relationship with academic performance. These findings suggest that tracking student intra¬ 
semester assessment submission times is predictive of academic achievement, but not an indicator of 
their self-reported level of organization. Flowever, the combination of tracking of submission times and 
targeted questioning within regular meta-learning tasks may allow academics to intervene and promote 
improved planning and organizational behaviour by their students. 
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It was difficult to verify the extent to which students prepared for learning from the access patterns of 
learning resources in this course. The course was run traditionally, with the online resources available 
prior to classes being confined to lecture notes, assessment information, and introductory videos for the 
practical classes. Access to lecture notes was often limited to a short time (a few days) prior to the 
scheduled lecture time, and consequently student access to these resources was not tracked. However, 
student access to the introductory practical videos provides some insight into their preparedness. Over 
half the students accessed these resources prior to their first practical class, most immediately prior to 
the class, whilst 65% of all students accessed them at some time during the semester (Figure 4). 
Considering this data in relation to the low number of students who reported engaging in forethought 
processes in the first meta-learning task, this finding suggests that students tend to under-report their 
planning and preparedness for learning, unless specifically prompted to articulate their forethought 
processes. As it has been demonstrated that the most effective self-regulated learners implement 
strategies from all phases of the self-regulation cycle (Cohen, 2012; Kitsantas, 2002; Schunk & Swartz, 
1993; Aregu, 2013; Cleary, Zimmerman, & Keating, 2006), further verification of the extent to which 
students engage in forethought strategies is needed. 


There is good evidence to suggest that prompting students to engage in meta-learning is beneficial to 
them, increasing their self-regulatory skills and deepening their understanding of their learning (Hattie, 
2009), hence the inclusion of meta-learning assessment tasks in our course design. The primary purpose 
and value of these tasks was to prompt students to undertake self-regulation, particularly to promote 
the use of learning strategies from the forethought and self-reflection phases. Strategies from these 
phases are considered more advanced, and using a combination of strategies from all three phases has 
been associated with higher academic achievement (Pintrich, 1995; Zimmerman, 2002; Kitsantas, 2002). 
In this study, the data generated from these tasks was utilized to develop an understanding of students' 
self-regulatory behaviour. However, although rich, the data is rather laborious to analyse and is limited 
by the accuracy of self-reporting. While calibration of this form of self-reporting by students and their 
self-regulatory behaviour has not been undertaken, other forms of self-reportage have shown widely 
differing levels of accuracy (Hadwin et al., 2007; Schmitz & Wiese, 2006). Comparison of the findings 
from the meta-learning data with the online access data from the learning management system 
suggests that students under-report the extent of their self-regulatory behaviour. However, student 
responses to the open-ended questions in the meta-learning tasks often included lengthy and detailed 
descriptions on the nature and types of strategies employed, and there is close alignment in the cluster 
analysis on the lecture recordings with the student reports. Together, this suggests that although under¬ 
reporting, students are not necessarily inaccurate in the types of strategies they describe. 

Due to time constraints, analysis of the meta-learning data was confined to categorization and 
quantification of the self-regulatory strategies that students reported, but the lack of correlation 
between individual categories of strategies with performance found here are both contrary to literature 
and to our previous research using more detailed analysis methods (Kitsantas, Winsler, & Huie, 2008; 
Ogiji et al., submitted). Analysis did show, however, that student use of strategies from multiple phases 
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of the self-regulatory cycle was positively correlated with academic performance. While the meta¬ 
learning data may allow an evaluation of the categories of strategies they use, simply categorizing and 
quantifying appears to be insufficient to give an accurate evaluation. Indeed, we and others 
(Nandagopal & Ericsson, 2012; Ogiji et al., submitted) have shown that academic performance is not 
simply related to the use of, or particular categories of learning strategies, but rather to the quality, 
diversity, and timing of those learning strategies. Thus, as Winne (2010) suggests, it is likely that higher 
performing students have more appropriate contingencies, evaluating and adapting their behaviour to 
suit the task. For example, using Winne's terminology, IF a high performing student finds that reading 
the textbook does not produce the desired learning outcomes, they might TFIEN seek information from 
peers or academics; while a lower performing student's THEN behaviour might be to read the text book 
for longer (Winne, 2010). These differing adaptations may not be discernible through simple 
categorization or quantification of strategies. Clearly, all students are capable of self-regulating their 
learning, but the quality, quantity, and timing of the self-regulatory processes differs between learners 
(Cohen, 2012). Potentially, utilizing learning analytics to evaluate the data generated by the course 
learning management system may offer a fast method to gauge students' self-regulation and progress as 
learners, to supplement the meta-learning findings, but the way in which such data reflects the self- 
regulatory behaviour of students needs to be understood. 

4.1 Future directions 

The methodologies used to research self-regulated learning are moving away from reliance on uni- 
modal data sources, such as self-report questionnaires (Pintrich, 2004), think-aloud protocols (Greene, 
Robertson, & Costa, 2011), or computer logs (Aleven, Roll, McLaren, & Koedinger, 2010) toward multi¬ 
modal data sources (Ben-Eliyahu & Bernacki, 2015). Recent research has attempted to triangulate 
multiple sources, such as case studies, interviews, and observations (Lichtinger & Kaplan, 2015), or 
combined quantitative surveys with qualitative open-response questions (McCardle & Hadwin, 2015). 
However, these approaches tend to still be bound in a single paradigm (observational, self-report, or 
computer logs) rather than bridging diverse data sources such as the open-ended meta-learning 
questions and learning management site interactions presented in this study. Admittedly, this study has 
used narrow slices from these different methodologies as an early exploration of the degree to which 
such data can be triangulated. As a next step, we plan to investigate the utility of chronologically 
sequencing multiple computer logs with student responses to multiple meta-learning tasks. This 
juxtaposition will help us to determine the temporal and sequential characteristics of students' self- 
regulated learning behaviours and meta-cognition, both at the micro-analysis level during tasks, and 
across the long-term changes that occur during a semester or over semesters. Such approaches will 
hopefully reveal both the critical points of change in self-regulated learning, and the degree to which 
such changes are retained. 

However, such "tracer"-like approaches to the analysis of open-ended responses and computer log data 
lend themselves primarily to the detailed analysis of small numbers of purposefully selected students 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


151 


JOURNAL OF LEARNING ANALYTICS 


S8LAR 

SOCIETY 1 LEARNING 
ANALYTICS RESEARCH 

(2015). Know Thy Student! Combining Learning Analytics and Critical Reflections to Increase Understanding of Students' Self-Regulated 
Learning in an Authentic Setting. Journal of Learning Analytics, 2(1), 134-155. 

(e.g., subsets of students with high and low academic achievement profiles). To move into the realm of 
large-scale analysis of such rich data sets and patterns of temporal and sequential relationships, 
automated methods of text analysis would be required. As an early step in this direction, basic statistical 
methods in natural language processing (such as word counts and the contexts in which words occur) 
would be used to triangulate the computer-assisted analysis of meta-learning responses with the human 
inductive thematic analysis of the same responses (Sherin, 2013). By approaching the same data from 
two analytical approaches, an enriched model of the data is more likely to be revealed. This improved 
model might then be used to train and test algorithms in a machine learning approach to achieve at 
least semi-automated analysis of large banks of meta-learning responses in conjunction with computer 
logs, to predict academic outcomes (e.g., Ghiasinejad & Golden, 2013). Potentially, predictive models 
might assist students who are struggling to self-evaluate accurately, and help students focus on 
metacognitive and self-regulatory behaviours appropriate to their situation. 

5 CONCLUSION 

In recent years, much of the research on self-regulation utilizing online tools has supplemented data 
from learning management systems with other data collection methods, including computer traces, 
think-aloud protocols, diaries of studying, direct observation, and microanalyses (Bannert, Reimann, & 
Sonnenberg, 2013; Ferreira, Simao, & da Silva, 2014; Hadwin et al., 2007; Perry & Winne, 2006; Schmitz 

6 Wiese, 2006), or has focused on courses offered primarily or entirely online (Lynch & Dembo, 2004). 
However, in the non-research, on-campus environment when these additional tools are not available, 
what can the data that is available tell us? It would appear that some aspects of the currently available 
data may be predictive of academic performance, but the relationship between this data the self¬ 
regulation and progress of the students is complex. Potentially though, the combination of tracking 
student engagement with online resources, intra-semester assessment submission times and targeted 
questioning within regular meta-learning tasks may allow coalface academics both to increase their 
understanding of student self-regulatory behaviour and to create interventions to promote 
improvements student learning outcomes. 

REFERENCES 

Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2010). Automated, unobtrusive, action-by-action 
assessment of self-regulation during learning with an intelligent tutoring system. Educational 
Psychologist 45(4), 224-233. 

Aregu, B. B. (2013). A study of self-regulated learning strategies as predictors of critical reading. 
Educational Research and Reviews, 8, 1961-1965. 

Bannert, M., Reimann, P., & Sonnenberg, C. (2013). Process mining techniques for analysing patterns 
and strategies in students' self-regulated learning. Metacognition and Learning, 9(2), 161-185. 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


152 


JOURNAL OF LEARNING ANALYTICS 


S 0 LAR 


(2015). Know Thy Student! Combining Learning Analytics and Critical Reflections to Increase Understanding of Students' Self-Regulated 
Learning in an Authentic Setting. Journal of Learning Analytics, 2(1), 134-155. 


Ben-Eliyahu, A., & Bernacki, M. L. (2015). Addressing complexities in self-regulated learning: A focus on 
contextual factors, contingencies, and dynamic relations. Metacognition and Learning, 10(1), 1- 
13. 

Boekaerts, M. (1997). Self-regulated learning: A new concept embraced by researchers, policy makers, 
educators, teachers, and students. Learning and Instruction, 7(2), 161-186. 

Bonestroo, W. J., & De Jong, T. (2012). Effects of planning on task load, knowledge, and tool preference: 
A comparison of two tools. Interactive Learning Environments, 20(2), 141-153. 

Cleary, T. J., & Zimmerman, B. J. (2004). Self-regulation empowerment program: A school-based 

program to enhance self-regulated and self-motivated cycles of student learning. Psychology in 
the Schools, 41(5), 537-550. 

Cleary, T. J., Zimmerman, B. J., & Keating, T. (2006). Training physical education students to self-regulate 
during basketball free throw practice. Research Quarterly for Exercise and Sport, 77(2), 251-262. 

Cohen, M. T. (2012). The importance of self-regulation for college student learning. College Student 
Journal, 46(A), 892-902. 

Dunlosky, J., & Rawson, K. A. (2012). Overconfidence produces underachievement: Inaccurate self- 

evaluations undermine students' learning and retention. Learning and Instruction, 22(A), 271- 
280. 

Ernst, H., & Colthorpe, K. (2007). The efficacy of interactive lecturing for students with diverse science 
backgrounds. Advances in Physiology Education, 31(1), 41-44. 

Ferreira, P. C., Simao, A. M. V., & Da Silva, A. L. (2014). Does training in how to regulate one's learning 
affect how students report self-regulated learning in diary tasks? Metacognition and Learning. 

doi:10.1007/si 1409-014-9121-3 

Ghiasinejad, S., & Golden, R. M. (2013). Modeling human coding of free response data. Computers in 
Human Behavior, 29(6), 2394-2403. 

Greene, J. A., Robertson, J., & Costa, L. J. C. (2011). Assessing self-regulated learning using think-aloud 
methods. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and 
performance (pp. 313-328). New York, NY: Routledge. 

Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., & Winne, P. H. (2007). Examining trace data to 
explore self-regulated learning. Metacognition and Learning, 2(2-3), 107-124. 

Flattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses related to achievement. 
Abingdon, UK: Routledge. 

Jackson, N. (2004). Developing the concept of metalearning. Innovations in Education and Teaching 
International, 41(A), 391-403. 

Kitsantas, A. (2002). Test preparation and performance: A self-regulatory analysis. The Journal of 
Experimental Education, 70(2), 101-113. 

Kitsantas, A., Winsler, A., & Huie, F. (2008). Self-regulation and ability predictors of academic success 
during college: A predictive validity study. Journal of Advanced Academics, 20(1), 42-68. 

Lichtinger, E., & Kaplan, A. (2015). Employing a case study approach to capture motivation and self¬ 
regulation of young students with learning disabilities in authentic educational contexts. 
Metacognition and Learning, 10(1), 119-149. 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


153 


JOURNAL OF LEARNING ANALYTICS 


S 0 LAR 


(2015). Know Thy Student! Combining Learning Analytics and Critical Reflections to Increase Understanding of Students' Self-Regulated 
Learning in an Authentic Setting. Journal of Learning Analytics, 2(1), 134-155. 


Luo, W., Paris, S. G., Hogan, D., & Luo, Z. (2011). Do performance goals promote learning? A pattern 
analysis of Singapore students' achievement goals. Contemporary Educational Psychology, 

36(2), 165-176. 

Lynch, R., & Dembo, M. (2004). The relationship between self-regulation and online learning in a 

blended learning context. The International Review of Research in Open and Distance Learning, 
5(2). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/189/271 

McCardle, L., & Hadwin, A. F. (2015). Using multiple, contextualized data sources to measure learners' 
perceptions of their self-regulated learning. Metacognition and Learning, 10(1), 43-75. 

Nandagopal, K., & Ericsson, K. A. (2012). An expert performance approach to the study of individual 
differences in self-regulated learning activities in upper-level college students. Learning and 
Individual Differences, 22(5), 597-609. 

Nota, L., Soresi, S., & Zimmerman, B. J. (2004). Self-regulation and academic achievement and resilience: 
A longitudinal study. International Journal of Educational Research, 41(3), 198-215. 

Ogiji, J., Colthorpe, K., Ainscough, L., Zimbardi, K., & Anderson, S. (submitted). Meta-learning assessment 
tasks provide insight into undergraduate students' self-regulatory learning behaviour. 

Perry, N. E., & Winne, P. H. (2006). Learning from learning kits: gStudy traces of students' self-regulated 
engagements with computerized content. Educational Psychology Review, 18(3), 211-28. 

Pintrich, P. R. (1995). Understanding self-regulated learning. New Directions for Teaching and Learning, 
63, 3-12. 

Pintrich, P. R. (2004). A conceptual framework for assessing motivation and self-regulated learning in 
college students. Educational psychology review, 16(4), 385-407. 

Postholm, M. B. (2011). Self-regulated learning in teaching: Students' experiences. Teachers and 
Teaching, 17(3), 365-382. 

Sandars, J., & Cleary, T. J. (2011). Self-regulation theory: Applications to medical education. AMEE Guide 
No. 58. Medical Teacher, 33(11), 875-86. 

Schmitz, B., & Wiese, B. S. (2006). New perspectives for the evaluation of training sessions in self- 

regulated learning: Time-series analyses of diary data. Contemporary Educational Psychology, 
31(1), 64-96. 

Schunk, D. H. (2005). Self-regulated learning: The educational legacy of Paul R. Pintrich. Educational 
Psychologist, 40(2), 85-94. 

Schunk, D. H. (2008). Metacognition, self-regulation, and self-regulated learning: Research 
recommendations. Educational Psychology Review, 20(4), 463-467. 

Schunk, D. H., & Swartz, C. W. (1993). Goals and progress feedback: Effects on self-efficacy and writing 
achievement. Contemporary Educational Psychology, 18(3), 337-354. 

Sherin, B. (2013). A computational study of commonsense science: An exploration in the automated 
analysis of clinical interviewdata . Journal of the Learning Sciences, 22(4), 600-638. 

Turner, J. E., & Husman, J. (2008). Emotional and cognitive self-regulation following academic shame. 
Journal of Advanced Academics, 20(1), 138-173. 

Vilalta, R., Giraud-Carrier, C., & Brazdil, P. (2010). Meta-learning-concepts and techniques. Data mining 
and knowledge discovery handbook. New York, NY: Springer. 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


154 


JOURNAL OF LEARNING ANALYTICS 


S 0 LAR 

SOCIETV Itr LEARNING 


(2015). Know Thy Student! Combining Learning Analytics and Critical Reflections to Increase Understanding of Students' Self-Regulated 
Learning in an Authentic Setting. Journal of Learning Analytics, 2(1), 134-155. 


Ward, J. H., Jr. (1963). Hierarchical grouping to optimize an objective function. Journal of the American 
Statistical Association, 58(301), 236-244. 

Winne, P. H. (1996). A metacognitive view of individual differences in self-regulated learning. Learning 
and Individual Differences, 8(4), 327-353. 

Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B. J. 
Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement: 
Theoretical perspectives, 2nd ed. (pp. 153-189). New York, NY: Lawrence Erlbaum. 

Winne, P. H. (2010). Improving measurements of self-regulated learning. Educational Psychologist, 
45(4), 267-276. 

Winters, T. (2011). Facilitating meta-learning in art and design education. International Journal of Art & 
Design Education, 30, 90-101. 

Zimmerman, B. J. (ed.) (2000). Attaining self-regulation: A social cognitive perspective. Burlington, MA: 
Elsevier Academic Press. 

Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 
64-70. 

Zimmerman, B. J. (2006). Development and adaption of expertise: The role of self-regulatory processes 
and beliefs. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge 
handbook of expertise and expert performance. Cambridge, UK: Cambridge University Press. 

Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, 

methodological developments, and future prospects. American Educational Research Journal, 
45(1), 166-183. 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


155 


